# Efficient Inference Optimization

AM-Thinking-v1 is a text generation model based on the GGUF format, suitable for various natural language processing tasks.

The Exllamav3 quantized version of Qwen3-235B-A22B, offering multiple quantization options to optimize model size and performance.

Llama-3.1-Nemotron-Nano-4B-v1.1 is a compressed and optimized large language model based on Llama 3.1, focusing on inference and dialogue tasks, supporting 128K context length, and compatible with a single RTX GPU.

Falcon-H1 is an efficient hybrid architecture language model developed by TII, combining the advantages of Transformers and Mamba architectures, supporting English and multilingual tasks.

Falcon-H1 is a hybrid architecture language model developed by the UAE's Technology Innovation Institute, combining the strengths of Transformers and Mamba architectures, supporting multilingual processing.

Quantized version of OpenThinker2-7B, using llama.cpp for quantization, suitable for text generation tasks.

The NVIDIA Nemotron-H-8B-Base-8K is a large language model (LLM) developed by NVIDIA, designed to generate completions for given text fragments. The model adopts a hybrid architecture primarily composed of Mamba-2 and MLP layers, incorporating only four attention layers. It supports a context length of 8K and covers multiple languages including English, German, Spanish, French, Italian, Korean, Portuguese, Russian, Japanese, and Chinese.

An inference and dialogue model optimized from Meta Llama-3.1-8B-Instruct, supporting 128K context length, balancing efficiency and performance

This is the GGUF quantized version of Google Gemma 3B model, suitable for local inference and supports text generation tasks.

This model is a GGUF format version converted from Google's Gemma 3 27B IT model, suitable for local inference.

This is a GGUF format text ranking model converted from the BAAI/bge-reranker-v2-m3 model, supporting multilingual text embedding inference.

The FormatClassifier model categorizes web content into 24 classes based on URL and text content.

A topic classification model fine-tuned based on gte-base-en-v1.5, capable of classifying web content into 24 categories

PLaMo 2 8B is an 8-billion-parameter hybrid architecture language model developed by Preferred Elements, supporting English and Japanese text generation.

PLaMo 2 1B is a 1-billion-parameter model developed by Preferred Elements, pretrained on English and Japanese datasets, featuring a hybrid architecture combining Mamba and sliding window attention mechanisms.

A QA model fine-tuned on SQuAD 2.0 dataset based on ModernBERT-large, supporting long-context processing

A statically quantized model based on Menlo/Ichigo-llama3.1-s-instruct-v0.4, offering multiple quantization versions to suit different hardware requirements.

DeepSeek-V2-Lite is a cost-efficient Mixture of Experts (MoE) language model with a total of 16B parameters and 2.4B active parameters, supporting a 32k context length.

This model is fine-tuned based on meta-llama/Meta-Llama-3-8B-Instruct, specifically designed for function calling and JSON mode.

MiniCPM-MoE-8x2B is a Transformer-based Mixture of Experts (MoE) language model, designed with 8 expert modules where each token activates 2 experts for processing.

DeciLM 6B-Instruct Model is an English language model specifically designed for short-format instruction following, trained using LoRA fine-tuning technology based on DeciLM 6B